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Another criterioa that may be employed to select a subset 
of the represeotative images is based oa the length of the 
soeoe from which the representative image was taken. For 
example, only represeotative images taken form the longest 
of the scenes io the video program may be retained since 5 
these scenes arc presumably ihc most significant For 
example, a video program of a speaker making a presenta- 
tion before an audience may contain many longer scenes of 
the speaker interrupted by occasional brief shots of the 
audience. If the representative frames from only the longest jq 
scenes are retained, then representative frames of the 
speaker will be retained while the representative frames of 
the audience will be eliminated. 

In some cases it may be desirable to eliminate represen- 
tative frames associated with advertisements if the video 15 
programs are recorded from commercial television, for 
example. These representative frames may be easily 
removed because most commercials are either not captioned 
or are captioned in a mode different from the remainder of 
the video program. Accordingly, the change in caption 20 
modes can be used to detect advertisements which are to be 
omitted from the HTLM transcript. 

Another format that may be used to display HTML 
pictorial transcripts takes advantage of a mechanism known 
as server push, which is available on recent versions of the 25 
Netscape browser. Server push allows an HTML page to 
undergo changes while it is being viewed. This browser 
feature can be used to maintain a suitable page layout (e.g., 
a layout having a maximum number of images) without 
needing to eliminate sequentially occurring images. This 30 
feature, which could also be implemented using Java 
Animations, will be illustrated with reference to FIG. 2. FIG. 
2(a) shows an HTML page of a pictorial transcript which has 
three sequential images 1, 2, and 3, without any intervening 
captions. However, suppose the page formal which is 35 
selected dictates that only one image is to be displayed on a 
page, as in FIG. 2(6). Server push may be used display the 
images as shown in FIGS. 2(c)-2(e). When the page is first 
displayed at time tl in FIG. 2(c). only the first image is 
displayed. Using server push, the second image can be 40 
displayed at a later time t2 (e.g. one second later), as shown 
in FIG. 2(d}. At yet a later time t3 the third image can be 
displayed, as in FIG. 2(e). Moreover, if the network band- 
width and client and server throughput are sufficiently high, 
video shorts (real-time playback) can be made to appear at 45 
the caption breaks. 

In many cases a user will not be interested in viewing the 
HTML pictorial transcript in a sequential manner. Rather, 
the user may be only interested in those portions of the 
transcript that pertain to a particular topic. In such cases the 50 
user may wish to perform a keyword search of the HTML 
pictorial transcript. The HTML generator can perform the 
search on the closed-cap lioned text and emphasize those 
portions of the transcript that contain the keyword. For 
example, images that appear immediately prior to and after 55 
the occurrence of a keyword may be displayed at full 
resolution while other images may be di^layed at a smaller 
size and resolution. FIG. 3 shows an example of this formal 
after a search for the word "Tokyo." The smaller images may 
be hypertext links to the corresponding full sized images. In 60 
some cases, particularly for large HTML pictorial 
transcripts, hypertext anchors may be used in place of the 
small images to reduce bandwidth. If the keyword appears 
more than once in the transcript, a chain of links may be 
created among the individual occurrences of the word. For 65 
example, in FIG. 3, the arrows denote a link to other 
occurrences of the term "Tokyo." The HTML pictorial 
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transcript may also include hypertext anchors to other 
HTML documents which contain material supplementary to, 
or related to, the information in the transcript. 

The HTML generator may create an index page for the 
HTML pictorial transcript using conventional methods such 
as linguistic techniques, for example. RG. 4 shows one 
example of such an index page, which may be located as the 
first page of the document. The index may contain links to 
the individual pages of the transcript. The index may also 
include other information such as index terms obtained by 
linguistic analysis techniques. In FIG. 4, a portion of the 
index is available for the user to list additional keywords to 
serve as index terms. The index terms may be hypertext links 
to those locations in the transcript where the terms appear. 

Similar to the HTML documents previously discussed, 
HTML pictorial transaipts in which the representative 
frames are each associated with a corresponding audio 
segment may be arranged in a variety of different formats. 

For example, the individual representative frames may 
serve as links to the audio segment. Alternatively, anchors 
may be associated with the representative frames. By click- 
ing on the anchors the respective audio segments are played. 

It will be appreciated that those skilled in the art will be 
able to devise numerous arrangements which, although not 
explicitly shown or described herein, embody the principles 
of the invention. Accordingly, all such alternatives, modifi- 
cations and variations which fall within the spirit and broad 
scope of the appended claims will be embraced by the 
principles of the invention. For example, while the invention 
has been described as electronic data representing a con- 
densed version of a video program that is formatted as an 
HTML document for the World Wide Web, the invention is 
more generally applicable to such data that is formatted in 
any hypertext language suitable for electronic retrieval on a 
computer or over a communications network. 

We claim: 

1. A method for automatically providing a compressed 
rendition of a video program in a format suitable for 
eieclronic searching and retrieval, said method comprising 
the steps of: 

receiving electronic data representing a condensed ver- 
sion of a video program, said video program having a 
video component and a second information-bearing 
media component associated therewith, said electronic 
data representation including a representative frame 
from each segment of the video component of the video 
program and a portion of said second media component 
associated with said segment; 

automatically transforming said electronic data represen- 
tation into a hypertext format to form a hypertext 
pictorial transcript; and 

recording said hypertext pictorial transcript in an elec- 
tronic medium. 

2. The method of claim 1 wherein said second media 
component is closed -caption text. 

3. The method of claim 1 wherein said second media 
component is audio and wherein said portions of said audio 
component are represented by hypertext anchors. 

4. The method of claim 1 wherein said hypertext format 
is hypertext markup language. 

5. The method of claim 1 wherein said hypertext pictorial 
Uanscript comprises a plurality of hypertext pages, each of 
said pages having a prescribed maximum size. 

6. The method of claim 5 wherein said prescribed maxi- 
mum size is determined by a maximum number of frames 
per page. 
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7. The method of claim 5 wherein said plurality of 
hypertext pages are intercoonected by hypertext links. 

8. The method of claim 1 wherein said hypertext pictorial 
transcript has at least one standard page layout 

9. The method of claim 8 wherein said standard page 5 
layout includes a subset of representative frames selected by 

at least one criterion that reduces bandwidth. 

10. The method of claim 9 wherein said criterion removes 
substantially redundant representative frames. 

U. The method of claim 9 further comprising the step of lO 
replacing said substantially redundant frames with hypertext 
anchors. 

12. The method of claim 9 wherein said criterion removes 
alternating ones of sequentially occurring representative 
frames. 15 

13. The method of claim 9 wherein said criterion removes 
representative frames below a prescribed image size. 

14. The method of claim 9 wherein said criterion removes 
representative frames above a prescribed image size. 

15. The method of claim 9 wherein said criterion removes 20 
representative frames that differ from other representative 
frames by less than a prescribed amount. 

16. The method of claim 9 wherein said criterion removes 
representative frames taken &om segments below a thresh* 
old length. 25 

17. The method of claim 9 wherein said criterion removes 
representative frames taken from advertisements. 

18. The method of claim 1 wherein said hypertext picto- 
rial transcript has a user customizable page layout. 

19. The method of claim 1 further comprising the steps of 30 
generating and recording an index page to the hypertext 
pictorial transcript. 

20. The method of claim 19 wherein said index page 
includes links to individual pages of the hypertext pictorial 
U-anscript. 
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21. The method of claim 20 wherein said index page 
includes hypertext index terms indexed to pages of the 
hypertext pictorial U'anscript. 

22. The method of claim 1 wherein said hypertext picto- 
rial transcript has a plurality of standard page layouts 
selectable by a user. 

23. The method of claim 1 further comprising the step of 
transmitting said hypertext pictorial transcript over a com- 
munications network. 

24. The method of claim 23 wherein said network is the 
World Wide Web. 

25. The method of claim 1 wherein said hypertext picto- 
rial transcript comprises a plurality of hypenext pages. 

26. The method of claim 25 wherein said hypertext pages 
arc divided based on topic segmentation. 

27. The method of claim 25 wherein said hypertext pages 
are divided based on a change in closed-caption format, 

28. A method for automatically providing a compressed 
rendition of a video program in a format suitable for 
electronic searching and retrieval, said method comprising 
the steps of: 

receiving electronic data representing a condensed ver- 
sion of a video program, said video program having a 
video component and a second information-bearing 
media component associated therewith, said electronic 
data representation including a representative frame 
from each segment of the video component of the video 
program and a portion of said second media component 
associated with said segment; 

automatically transforming said electronic data represen- 
tation into a hypertext format to form a hypertext 
pictorial transcript; and 

recording said hypertext pictorial transcript in an elec- 
tronic medium. 

***** 



